Sim-to-Real Transfer of Soft Robotic Navigation Strategies That Learns From the Virtual Eye-in-Hand Vision

To steer a soft robot precisely in an unconstructed environment with minimal collision remains open challenge for robots. When the environments are unknown, prior motion planning navigation may not always be available. This paper presents novel Sim-to-Real method to guide cable-driven static under Simulation Open Framework Architecture (SOFA). The scenario aims resemble one of steps during simplified transoral tracheal intubation process where robotic endotracheal tube is guided upper trachea-larynx location by flexible video-assisted endoscope/stylet. In SOFA, we employ quadratic programming inverse solver obtain collision-free strategies endoscope/stylet manipulation based on model and encode virtual eye-in-hand vision. Then, associate anatomical features recognized vision joint space using closed-loop nonlinear autoregressive exogenous (NARX) network. Afterward, transfer learned knowledge prototype, expecting it navigate desired spot new phantom automatically its only. Experiment results indicate that our can efficaciously through unstructured according what has from environment. show average R-squared coefficient between NARX-forecasted SOFA-referenced robot's cable prismatic 0.963 0.997, respectively. visions also demonstrate good alignment tip glottis.